Backfill Project Sidewalk contributors from the NSF annual reports#1421
Merged
Conversation
Project Sidewalk now consumes /api/v1/projects/sidewalk/people/ as its
single source of truth for who contributed to PS. That endpoint is driven
by ProjectRole rows, so a contributor without one is invisible to Sidewalk
even if they have a member page.
Diffing the four NSF Crowd+AI annual report participant spreadsheets
(award #2125087, Y1-Y4 = Oct 2021 - Sep 2025) against the live API found
115 participant rows = 74 unique people, of which 45 were already linked.
The other 29 split into two gaps:
- 10 had a member page but no Sidewalk ProjectRole (Y1 undergrads plus
the faculty/collaborators)
- 17 had no Person record at all -- every one from the UIC subaward
Adds seed_sidewalk_participants, modelled on seed_project_aliases (#944):
a baked-in seed table plus a create-only handler wired into
docker-entrypoint.sh as step 4.10d. There's no shell or DB access on the
test/prod servers, so a startup one-shot is the only path.
Create-only is the important property, since this runs on every container
start long after editors have tuned these records in /admin: it never
modifies an existing Person, never adds a Position to anyone who has one,
and never touches an existing ProjectRole. A link-only entry whose Person
can't be found is skipped with a warning rather than recreated, so a rename
or merge can't spawn a duplicate.
Dates come from the NSF reporting periods (Oct 1 - Sep 30), the finest
granularity the spreadsheets offer; consecutive appearances merge into one
role. Headshots are out of scope -- Person.save() supplies the Star Wars
fallback and real ones get uploaded via /admin, which is what sets the
Cropper.js crop box.
Also adds Title.PROJECT_COORDINATOR for the UIC project coordinator, whose
role "Research Scientist" overstated. Note that a new Title REQUIRES a
TITLE_ORDER_MAPPING entry -- admin/position_admin.py sorts the dropdown by
it and would KeyError otherwise -- so a test now pins every Title as mapped.
Two data corrections the spreadsheets got wrong: Lauren Frame is modelled
with two Positions (undergrad through May 2024, then UIC's Occupational
Therapy Doctorate from Aug 2024; the sheets had her Y2/Y3 status
backwards), and Xia Su and Lisa Orii are deliberately excluded despite
appearing in the reports.
Tests cover idempotency, that admin edits survive a re-run, the held-back
sentinel for unconfirmed titles, the mid-grant title change, and that
seeded people surface through the real API route.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
jonfroehlich
added a commit
that referenced
this pull request
Jul 22, 2026
Ships the seed_sidewalk_participants backfill plus the Pillow 12.3.0 bump. Note: this release has NOT been verified on makeabilitylab-test -- the test host's auto-deploy has been rebuilding from a source tree stuck at 578d643 since Jul 18, so nothing merged since then has landed there. The backfill was instead validated against live prod data (all 27 seed entries checked for existence/duplicates) plus the full test suite. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Project Sidewalk now consumes
/api/v1/projects/sidewalk/people/as its single source of truth for who contributed to PS. That endpoint is driven byProjectRolerows, so a contributor without one is invisible to Sidewalk — even if they have a member page.Diffing the four NSF Crowd+AI annual report participant spreadsheets (award #2125087, Y1–Y4 = Oct 2021 → Sep 2025) against the live API found 115 participant rows = 74 unique people. 45 were already linked correctly; the other 29 split into two gaps:
ProjectRolePersonrecord at allWhat
Adds
seed_sidewalk_participants, modelled onseed_project_aliases(#944): a baked-in seed table plus a create-only handler, wired intodocker-entrypoint.shas step 4.10d. There is no shell or DB access on the test/prod servers, so a startup one-shot is the only path for this.Create-only is the load-bearing property, since this runs on every container start long after editors have tuned these records in
/admin. It never modifies an existingPerson, never adds aPositionto anyone who already has one, and never touches an existingProjectRole. A link-only entry whosePersoncan't be found is skipped with a warning rather than recreated, so a rename or merge can't spawn a duplicate.Dates come from the NSF reporting periods (Oct 1 – Sep 30) — the finest granularity the spreadsheets offer, since they record which annual report someone appeared in, not the day they started. Consecutive appearances merge into a single role.
Also adds
Title.PROJECT_COORDINATORfor the UIC project coordinator, whose role "Research Scientist" overstated. Worth knowing for future contributors: a newTitlerequires aTITLE_ORDER_MAPPINGentry —admin/position_admin.pysorts the dropdown by it and wouldKeyErrorotherwise — so a test now pins everyTitleas mapped. No migration is committed (website/migrations/is gitignored per-environment); the entrypoint'smakemigrations/migrategenerates the choices-onlyAlterFieldon deploy, leaving the column untouched.Data judgment calls
Positions — undergrad through May 2024, then UIC's Occupational Therapy Doctorate from Aug 2024. The NSF sheets had her Y2/Y3 status backwards (Graduate in Y2, Undergraduate in Y3).Person.save()supplies the Star Wars fallback; real headshots get uploaded via/admin, which is what sets the Cropper.js crop box (Would be so nice to get an instant preview of an image that you add for cropping #1299). 16 are still being collected.Testing
python manage.py test website --settings=makeabilitylab.settings_test→ 620 passed, 8 skipped. 12 new tests cover idempotency, that admin edits survive a re-run, the held-back sentinel for unconfirmed titles, the mid-grant title change, the new Title, and that seeded people surface through the real API route.Dry run against a local DB:
18 positions because Lauren has two. On prod expect 17 people, 18 positions, 27 roles — the extra 9 roles are the link-only people the local dev DB doesn't have.
No UI changes, so no Pa11y run or screenshots (data-only).
Follow-ups (not in this PR)
ProjectRoledates don't cover the years they were reported — Michael Duan, Devon Snyder, Dylan Bunarto, Manjiri Dehankar./adminedits; they already appear in the API, only the dates are wrong.🤖 Generated with Claude Code